Correspondence Learning for Deep Multi-Modal Recognition and Fraud Detection

Deep learning-based methods have achieved good performance in various recognition benchmarks mostly by utilizing single modalities. As different modalities contain complementary information to each other, multi-modal based are proposed implicitly utilize them. In this paper, we propose a simple technique, called correspondence learning (CL), which explicitly learns the relationship among multiple The data samples randomly mixed samples. If from same sample (not mixed), then they positive correspondence, and vice versa. CL is an auxiliary task for model predict expected extract modality check achieve better representations tasks. work, first validate method including CMU Multimodal Opinion-Level Sentiment Intensity (CMU-MOSI) Opinion Emotion (CMU-MOSEI) sentiment analysis datasets. addition, fraud detection using learned To additional usage, collect dataset real-world reverse vending machines.